Information Bottleneck Theory on Convolutional Neural Networks

Recent years, many researches attempt to open the black box of deep neural networks and propose a various theories understand it. Among them, information bottleneck (IB) theory claims that there are two distinct phases consisting fitting phase compression in course training. This statement attracts attentions since its success explaining inner behavior feedforward networks. In this paper, we employ IB dynamic convolutional (CNNs) investigate how fundamental features such as layer width, kernel size, network depth, pooling layers multi-fully connected have impact on performance CNNs. particular, through series experimental analysis benchmark MNIST Fashion-MNIST, demonstrate is not observed all these cases. shows us CNNs rather complicated than